Deep Boltzmann Machines as Feed-Forward Hierarchies
نویسندگان
چکیده
The deep Boltzmann machine is a powerful model that extracts the hierarchical structure of observed data. While inference is typically slow due to its undirected nature, we argue that the emerging feature hierarchy is still explicit enough to be traversed in a feedforward fashion. The claim is corroborated by training a set of deep neural networks on real data and measuring the evolution of the representation layer after layer. The analysis reveals that the deep Boltzmann machine produces a feed-forward hierarchy of increasingly invariant representations that clearly surpasses the layer-wise approach.
منابع مشابه
Learning Feature Hierarchies with Centered Deep Boltzmann Machines
Deep Boltzmann machines are in principle powerful models for extracting the hierarchical structure of data. Unfortunately, attempts to train layers jointly (without greedy layerwise pretraining) have been largely unsuccessful. We propose a modification of the learning algorithm that initially recenters the output of the activation functions to zero. This modification leads to a better condition...
متن کاملA Machine Learning Approach for Abstraction Based on the Idea of Deep Belief Artificial Neural Networks
In a time-critical world knowledge at the right time might decide everything. However, storing data does not correspond with understanding the knowledge it contains. Thus, solutions capable of learning problem statements and gathering knowledge from huge amounts of data, be it structured or unstructured, are required. This is where computational intelligence and the introduced approach apply: w...
متن کاملResearch on the Classification Ability of Deep Belief Networks on Small and Medium Datasets
Recent theoretical advances in the learning of deep artificial neural networks have made it possible to overcome a vanishing gradient problem. This limitation has been overcome using a pre-training step, where deep belief networks formed by the stacked Restricted Boltzmann Machines perform unsupervised learning. Once a pre-training step is done, network weights are fine-tuned using regular erro...
متن کاملConstructing modular architectures with Boltzmann Machines
In this paper, I discuss how the Boltzmann Machine framework can be used for modeling hybrid and modular architectures. The advantage is that it provides a rm theoretical basis for the derivation of learning rules in the presence of lateral inhibition, feed-forward and feedback .
متن کاملDeterministic learning rules for boltzmann machines
It is shown that by introducing lateral inhibition in Boltzmann Machines (BMs), hybrid archi-tectures involving diierent computational principles, such as feed-forward mapping, unsupervised learning and associative memory, can be modeled and analysed. This is of great advantage for getting a better understanding of the capability of the Boltzmann Machine and for the study of hybrid architecture...
متن کامل